Systems and methods for providing information and performing task

ABSTRACT

Systems, methods, and apparatus for presenting information and performing a task using an electronic device. In some aspects, a device shows content items when a gaze or a shaking act plus a gaze are detected. In some aspects, a device performs a task when a name, a code, and the task are detected in voice input. In some aspects, a user communicates with a selected vehicle via a user device.

CROSS REFERENCE TO RELATED APPLICATION

This is a continuation-in-part of U.S. application Ser. No. 17/235,862,filed Apr. 20, 2021, which is a continuation-in-part of U.S. applicationSer. No. 16/779,676, filed Feb. 3, 2020, which is a division of U.S.application Ser. No. 15/917,625, filed Mar. 10, 2018, which is acontinuation-in-part of U.S. application Ser. No. 15/802,427, filed Nov.2, 2017, which is a continuation of U.S. application Ser. No.15/494,464, filed Apr. 22, 2017, which is a division of U.S. applicationSer. No. 14/217,486, filed Mar. 18, 2014, now U.S. Pat. No. 9,671,864,granted Jun. 6, 2017.

BACKGROUND Field of Invention

This invention relates to providing information and performing a task,more particularly to presenting information and performing a task at adevice after receiving a voice input, detecting a gaze, and/or receivinga message from another device.

Description of Prior Art

When a smartphone is standby, its display may turn dark to save energy.Without user intervention, the smartphone would stay that way. In somecases, a user may not want to play with a standby phone, because he orshe may be busy doing other things. In some other cases when a user isnot busy, he or she may still be reluctant to awake a phone from standbystate, if there isn't anything interesting. In the latter scenario, auser may have time to take or view information, while a smartphone mayhave a blank screen ready to display and convey info. However, therelack convenient ways and incentives for a user to start it. As aconsequence, the phone may continue to be idle, while a user may justgaze at a dark empty screen, causing a waste of time for both the userand phone.

Accordingly, there exists a need to utilize idle time of a smart phoneand other electronic devices to present information to idling users.

Advertisements represent a major revenue source for many internetservice providers and internet companies. When users surf on theInternet or communicate with each other, however, most hold a rathernegative attitude towards advertisements, which often tend to presentcertain content in an intrusive, disruptive, obtrusive, or even rudemanner. Intrusive ads include unexpected pop-up, unwelcome or oversizedbanners, or annoying flashing objects or pictures. On the other hand,advertisements made to be less intrusive often end up being ignored orless effective due to a weak or subtle appearance. In both cases, eitherusers are offended, or ad effect is in doubt.

Thus, it is desirable to have a method and system which provideadvertising information in a less-intrusive but effective way. Becausean idle device sometimes means an idling user, it may be less intrusiveand probably more effective to present advertisements utilizing an idledevice in an unused time slot. But so far most internet advertisementsappear at a rather awkward time, competing with programs a user isrunning or annoying a user who is already busy enough.

Therefore once again, there exists a need to utilize idle time ofelectronic devices like smartphones or tablet computers to presentinformation. The idle time may be especially useful for showingadvertising items to idle users.

When a user utters a command to a device, the device performs a taskindicated in the command. However, if there are multiple devices, morethan one device may respond to the command, causing difficulties toperform the task.

When a user approaches a vehicle and wants to utter a command, the useroften has to search for an interface device (e.g., a microphone orkeypad), walk very close to the interface device, and then speak to it.It takes time for the user to find the interface device, and it is oftenawkward to get very close to the interface device when the vehicle isparked by the roadside.

Thus, there exists a need for a user to utter a command to a device orvehicle in a simple, convenient, and natural way.

Objects and Advantages

Accordingly, several main objects and advantages of the presentinvention are:

-   -   a). to provide an improved method and system for presenting        information and performing a task;    -   b). to provide such a method and system which target an idle or        standby device;    -   c). to provide such a method and system which monitor the gaze        direction of a user to determine when to present information and        when to stop a presentation;    -   d). to provide such a method and system which use a user input        such as shaking, tapping, or voice command plus a gazing act to        determine when to present information;    -   e). to provide such a method and system which perform a task        after detecting a voice input and/or a gazing act;    -   f). to provide such a method and system which perform a task at        a first device after a second device detects a voice input and        transmits instructions to the first device;    -   g). to provide such a method and system which receive a voice        command from a user via a user device and transmit the command        to a selected vehicle; and    -   h). to provide such a method and system which receive voice        input at a standby device when the voice input contains a        program name or vehicle name.

Further objects and advantages will become apparent from a considerationof the drawings and ensuing description.

SUMMARY

In accordance with the present invention, methods and systems aredisclosed for presenting information and performing a task using anelectronic device. In some embodiments, when a user gazes at an idlescreen of an idle device, indicating the user might not be engaged inanything, the device may take the opportunity to present news, updates,or other information. In some embodiments, when a user shakes, taps, orspeaks to a standby or idling device, and then looks at it, the devicemay combine the shaking, tapping, or speaking act with the gazing actand consider the combination as a predetermined command to showinformation on a screen. In some embodiments, a task is performed at adevice when a voice input includes a name, a code, and the task, a voiceinput includes a name and a gaze act is detected, or a user utters acommand to a another device for doing the task.

In some embodiments, a user communicates with a selected vehicle via auser device when the vehicle approaches the user. In some embodiments, auser utters a voice command to a standby and locked device. When thevoice command includes a name of a program or a selected vehicle, theprogram implements the command.

DRAWING FIGURES

FIG. 1 is an exemplary block diagram describing an embodiment inaccordance with the present invention.

FIG. 2 illustrates exemplary diagrams showing an embodiment involving auser and a device in accordance with the present invention.

FIGS. 3, 4 and 5 are exemplary flow diagrams showing respectiveembodiments in accordance with the present invention.

FIG. 6 illustrates exemplary diagrams showing another embodimentinvolving a user and a device in accordance with the present invention.

FIG. 7 is an exemplary flow diagram showing steps of the embodimentdepicted in FIG. 6 in accordance with the present invention.

FIG. 8 illustrates an exemplary diagram showing embodiments involving auser, a user device, a control device, and an application device inaccordance with the present invention.

FIG. 9 illustrates an exemplary diagram showing embodiments involving auser, a user device, and a vehicle in accordance with the presentinvention.

REFERENCE NUMERALS IN DRAWINGS 10 Sensor 12 Database 14 CommunicationNetwork 16 Processor 18 Processing Module 20 Sensor 22 Computer ReadableMedium 24 Sensor 30 Smartphone 32 Eye 36 Smartphone 38 Eye 40 User 42User Device 44 Control Device 46 Application Device 48 Vehicle 50Compartment 80 Client System 82 Service Facility 83 Service Center 100,102, 103, 104, 105,106, 108, 110, 112, 114, 116, 118, 120, 122, 124,126, 128, 130, 132, 133, 134, 136, 138, 140, 144, 146, 148, 150, 152 areexemplary steps.

DETAILED DESCRIPTION

The following exemplary embodiments are provided for complete disclosureof the present invention and to fully inform the scope of the presentinvention to those skilled in the art, and the present invention is notlimited to the schematic embodiments disclosed, but can be implementedin various types.

FIG. 1 is an exemplary block diagram of one embodiment according to thepresent invention. A client system 80 and service facility 82 areconnected via a communication network 14. Client 80 may represent anelectronic device, including but not limited to a desktop computer, ahandheld computer, a tablet computer, a wireless gadget (such as mobilephone, smart phone, smart watch, and the like), etc. Client 80 mayinclude a processor 16 and computer readable medium 22. Processor 16 mayinclude one or more processor chips or systems. Medium 22 may include amemory hierarchy built by one or more memory chips or storage moduleslike RAM, ROM, FLASH, magnetic, optical and/or thermal storage devices.Processor 16 may run programs or sets of executable instructions storedin medium 22 for performing various functions and tasks, e.g., playinggames, playing music or video, surfing and searching on the Internet,email receiving and transmitting, displaying advertisements,communicating with another device, sending a command to another device(e.g., turning on another device or controlling the operation of anotherdevice), etc. Client 80 may also include input, output, andcommunication components, which may be individual modules or integratedwith processor 16. Usually, client 80 may have a display with agraphical user interface (GUI). The display surface may also besensitive to touches, especially in the case of tablet computer orwireless gadget. Client 80 may also have a microphone and a voicerecognition component to detect and recognize audio input from a user.

Service facility 82 may include a processing module 18 and database 12.Module 18 may contain one or more servers and storage devices toreceive, send, store and process related data or information.

The word “server” indicates a system or systems which may have similarfunctions and capacities as one or more servers. Main components of aserver may include one or more processors, which control and processdata and information by executing software, logic, code, or carrying outany other suitable functions. A server, as a computing device, mayinclude any hardware, firmware, software, or a combination. In the mostcompact form, a server may be built on a single processor chip. In thefigure, module 18 may contain one or more server entities that collect,process, maintain, and/or manage information and documents, performcomputing and communication functions, interact with users, deliverinformation required by users, etc. Database 12 may be used to store themain information and data related to users and the facility. Thedatabase may include aforementioned memory chips and/or storage modules.

A communication network 14 may cover a range of entities such as theInternet or the World Wide Web, a local area network (LAN), a wide areanetwork (WAN), a metropolitan area network (MAN), a telephone network,an intranet, wireless, and other types of networks. Client 80 andfacility 82 may be connected to network 14 by various wired, wireless,optical, or other connections.

Client 80 may include a sensor 10 which tracks the eye of a user usingmature eye-tracking technologies. The sensor may be arranged very closeto the screen of a display and designed to obtain a picture of thefacial part of a user. The system may recognize whether a user's gaze isin such a direction that the eye sight may fall on the display screen ofclient 80. In other words, sensor 10 may be employed to determinewhether a user is looking at the screen of a device through properalgorithms. Sensor 10 may be built using imaging technologies, and theimage of a user's eye may be analyzed to decide which direction the useris looking at. Both visible and infrared light may be employed foreye-tracking. In the latter case, an infrared light source may bearranged to provide a probing beam.

Client 80 may also include a sensor 20 which functions as a motiondetector, which is well known in the art and employed at some devicesalready. Sensor 20 may be used to detect movement of an object outsidethe device. It may include a camera-like system to obtain images andthen recognize any movement through image analysis over a period oftime. As sensor 10 has imaging capabilities, sensor 10 may be arrangedto work both as an eye-tracking device and as a motion detector, whichis desirable when small size is required.

Furthermore, client 80 may contain a sensor 24 to detect its ownmovement by sensing acceleration, deceleration, and rotation. Sensor 24may employ one or multiple accelerometers, gyroscopes, and/or pressuresensors for performing various measurement tasks which may includedetecting device shaking, device vibration, user running, user walking,and so on.

FIG. 2 shows exemplarily one embodiment according to the presentinvention. The essence is to utilize sleeping devices to bring info toidle users. At Step 1 of the figure, a smartphone 30 is standby oridling, with a dark screen showing nothing. At Step 2, a user gazes atthe screen, reflected by an eye 32 looking at it. If the gazing timeelapses beyond a certain value, it may be interpreted as the user mighthave spare time and might be willing to view info presented on thescreen. Then at Step 3, the screen lights up and content items arepresented. The user may continue to look at the screen and view thecontent items, or turn his or her sight away from the screen. If theuser redirects the gaze direction to elsewhere for a certain period oftime, it may be deemed as not wanting to watch the content any more.Then the screen may turn dark and the smartphone may become idle orstandby again, as depicted at Step 4.

Content items presented on an idling device may include any category ofinformation such as breaking news, regular news, market updates,newly-arrived shared photos, email alert, text messages, video clips,advertisements, community events, sports, and so on. A user may choosewhat information may be presented. A user may also rely on a programand/or a service provider, which is connected to a device viacommunication networks, to arrange content items to be presented.

FIG. 3 is a schematic flow diagram illustrating one embodiment ofproviding information according to the present invention. The processstarts with Step 100, occurrence of an idle device, meaning no user isactively doing anything with it and the idle mode has been there for awhile. A device being idle or standby may indicate the device has beenin that state for some time, beyond a given period. Examples of idlingdevice may include a desktop computer or tablet computer running byitself for a certain period of time without any input from users, acomputer or tablet computer running on screen-saver mode, a cell phoneor smartphone in standby state, i.e., ready to receive incoming callswhile in a lower-power energy-saving state, or in general, a runningelectronic device with a lower or much lower power consumption settingand probably a blank screen if it has one, etc. Next, at Step 102, thedevice detects a user's gaze and analyzes whether the user looks at itsdisplay, by sensor 10 in FIG. 1 for example. At Step 103, if the userdoesn't gaze at the display, the device may enter Step 105, remaining inidle or standby status. If the device detects that the user has beenlooking at the display for a certain period of time and its idle time isbeyond a given value simultaneously, the device may be programmed tograsp the opportunity and present a content window at Step 104. The newwindow may show information which a user may prearrange or show contentitems received over the network or from the Internet, like news update,event update, real-time broadcast, etc. As the user isn't runninganything at the device, it doesn't interfere with the user's activity;and since the user is looking at the screen, content presented may havea good chance to catch his or her attention. Next at Step 106, if theuser moves sight away from the screen, indicating the user may beunwilling to watch it any longer, the content window may close at Step110, and the display may return to the previous blank setting. Then thedevice may go back to idle state at Step 132. If the user keeps watchingthe content or keeps an eye on the screen, the device may stay engagedat Step 108, and the content window may remain on the screen. Thecontent items may cover a wide range of subjects and may switch topicsaccording to prearranged schedules.

Aside from turning idle time into informative or entertaining sessions,an idle user may also mean an opportunity for presenting certain specialkinds of information. Take advertisements for instance. If anadvertisement is introduced in the middle of a program which a user isrunning, it may offend the user due to the intrusive and disruptivenature. But if an ad is brought in at the end of a program, a user mayprepare to leave or start another task, and thus may not have enoughtime or interest watching the ad, causing ineffectiveness of advertisingeffort. On the other hand, when a user is idle and is gazing at a blankscreen, appearance of ads on the screen may be less intrusive andprobably more acceptable and more effective. After all, the user hasnothing to do and the ads may get enough attention. Moreover, the ad mayhave a chance to take a full screen, particularly valuable for deviceshaving a small screen size such as smartphones. Ads presented onsmartphones always have size issues due to limited screen dimension andlower priority status relative to what a user is doing or watching.

FIG. 4 is a schematic flow diagram illustrating another embodiment ofpresenting content items according to the present invention. At Step112, a content window appears on a display. Occurrence of the window maybe triggered by a user's gaze, like what described above regarding theprocess in FIG. 3 . Content items may be chosen by service providers orpre-selected by a user, or combination of both. If a user likes thecontent and keeps watching it, content window may stay for a while. Butif the content items are not appreciated or a user wants to run anotherprogram, he or she may want to close the window right away. Thus at Step114, the user may take an action like pushing a button, tapping an iconon a touch-sensitive screen, or clicking on an object using a mouse.Then at Step 116, the content window shrinks to a much smaller size, orbecomes an icon on the display. The window is not completely gonebecause a user may want to revisit it at a later time. At Step 118, if auser clicks on the shrunk window or icon, the content window may resume,and the content items may come back at Step 120. The user may startwatching the previous content items, or play with the window to findmore things of interest. If a user ignores the shrunk window at Step118, the window may remain there for a given period of time and then goaway, causing no nuisance to a user. In the meantime, the screen mayreturn to the previous setting at Step 122. In the former case, after auser goes back to the content items at Step 120 and spends enough time,the user may close the window and reaches Step 122, resuming apreviously paused session.

Returning to Step 104 of FIG. 3 . When a user opens up a content windowby gaze, he or she may watch it continuously, or close it with ease.FIG. 5 shows a schematic flow diagram to illustrate the situation indetail. At Step 124, a window is created on a display and content itemsare shown to a user. Meanwhile, the gaze direction of the user ismonitored continuously. At Step 126, if it is detected that the userlooks away from the display for a given period of time, Step 130 isimplemented. The content window closes and the device may return to itsidle or standby state. If the user keeps watching the display, it goesfrom Step 126 to Step 128, and the window remains open and content itemsare presented and refreshed per schedule in place. To provideconvenience for a user, a cycle is designed, which consists of Step 126to 128, then back to Step 126, and then to Step 128 or 130. As a result,a user may watch content items presented by the display on and on, andmeanwhile the user may close the content window at any time by lookingaway from the display. Optionally, a user may reopen the window any timeby looking at the display or reopen the window by running certainapplication designed for such a purpose. Therefore, a user may choose towatch scheduled content or walk away from it easily and conveniently.

Referring back to FIG. 1 , sensor 20 may be employed to work togetherwith sensor 10. For instance, sensor 20 may detect the movement of auser. When a user approaches a device, sensor 20 may detect it and thenthe system may activate sensor 10 to detect the user's gaze direction.In other words, physical movement of a user may be considered as a userinput to control the device. In the meantime, the device may be designedto wake up from sleep state and return to standby state after sensor 20detects a given signal. Since a motion detector may consume less powerthan an eye-tracking sensor, it saves energy and extends the batterylife of a device.

Senor 24 may be used to save energy of a device too. For example, whensensor 24 detects that a device's position is unstable or changes in anunusual way, the device may be configured to turn off sensor 10. Thusunder such a circumstance, its display may remain blank or inscreen-saver mode even when it is gazed by a user.

In addition, sensor 24 may be used to design another embodiment. Forinstance, a user may want to take initiative to lighten up a darkdisplay and make use of standby or idle device in a simple andconvenient manner. Suppose a user is looking at a blank screen of astandby smartphone 36, maybe at a subway station. The user may want towatch something to kill time, but doesn't have any idea about what towatch. So the user may follow the exemplary steps illustrated in FIG. 6to start a content show which would be presented on the idling device.Let us assume shaking is selected as an input signal and a detector likesensor 24 is arranged to detect whether a device is shaken by a user ornot. At Step 1, the user may shake smartphone 36 a bit. The shaking actis caught by the detector, which may send a signal to trigger a sensingprocess to ascertain whether the user gazes at the phone. For instance,a circuitry may be configured such that shaking may activate a gazesensing system. Then at Step 2, the user may look at the phone screen oran eye 38 may gaze at it as shown in the figure, which is detected andnext at Step 3, content items may show up on the screen. The contentitems may be selected by a service provider, including topics likeinstant news, weather forecast, promotions nearby, ads, and so on. Thuswith a little shaking and some gazing, a user may get content itemspresented to him or her on an idle device instantly. Compared to thegaze-only scenario as described in FIGS. 2 and 3 , the embodiment inFIG. 6 gives another option to a user. It also avoids content showscaused by unintended gaze. Probably more important, the scheme savesenergy as a gaze sensing system may be off most of the time unlessgetting activated upon receiving shaking signals.

Besides shaking, there are many other acts or other physical movementswhich may be employed as the first step to work with a dark screen andto view content items on it. For instance, tapping, scribbling orsliding on a touch-sensitive screen, or tapping on certain area of adevice where sensitive sensors may be placed, may also be incorporatedas the first indicator that a user may want to watch something on anidle device. It may depend on a specific app or program to specify whatkind of physical move may be taken as an input for a device. If there ismore than one option, a user may select a method which may seem moreconvenient and effective.

FIG. 7 shows an exemplary flow diagram to illustrate the embodimentdepicted in FIG. 6 with more details. Assume that tapping is designatedas the first signal needed. At Step 133, a device is in idle or standbymode except a tap sensor. The tap sensor, e.g., sensor 24 in FIG. 1 , ispowered on to detect tapping act performed by a user. A qualifiedtapping may be one tap or two consecutive taps with finger or hand. AtStep 134, if no tapping is received, the device may stay in the originalstate, being idle or standby as at Step 140. If tapping is sensed, agaze sensor may start working to detect whether a user gazes at thedisplay at Step 136. Next at Step 138, if the user's sight is not on thedisplay within a given period of time, the device may go to Step 140,returning to idle or standby state. If the user's sight or gaze turns tothe display within a given period of time and the act lasts long enough,a content window may show up at Step 144. Then at Step 146, the gazesensor may continue to monitor the user's gaze direction. If a userdoesn't want to watch the content, his or her gaze may be directed toelsewhere away from the device. Then the content window may close atStep 150 and the device may go back to an idle or standby mode at Step152. If the user keeps watching the content, his or her gaze stays withthe device, and the content show may continue at Step 148.

Speech recognition and voice generation functions may be incorporated tomake a process easy and smooth. For example, after a content window isstaged by a user's gazing act, the window may be closed when a usersimply says “No”, if speech recognition technology is employed.Additionally, a content window may be arranged to show up quickly aftera user says a predetermined word like “info” or “content” and thenstarts looking at the screen. A device may also generate a short speechto describe an info session after a content window is presented.

When voice recognition and gaze detection are used together, only onedevice, which is gazed at, may respond to a user's voice instructions.Thus a user may give a voice command to a device exclusively andconveniently by speaking to and looking at it. Without gaze detection,multiple devices may react to a voice command and it may cause a chaoticscene. Without voice recognition, a gazing act may invoke a single andoften simple task only, which limits applications.

Two scenarios may exist, when voice recognition and gaze detection areused to enable interaction between a user and a device: A user may saycertain word or words and then look at a device or say certain word orwords and look at a device at the same time. The two actions, i.e.,speaking and gazing, in both scenarios may be arranged to cause a deviceto carry out one or more tasks. As aforementioned, a gazing act means auser gazes at a device for at least a certain time period. The one ormore tasks may be predetermined. For instance, it may be arranged that auser may say a given word or short sentence. The given word or sentencemay indicate a request for one or more tasks. Then, a device may carryout the one or more tasks. A user may also say one or more sentences todescribe a task and ask a device to do it verbally. A device may usevoice recognition techniques to analyze and interpret a user's voiceinput and obtain one or more tasks from the input.

The one or more tasks include presenting certain content items on ascreen or via a speaker, turning on a device from standby or power-offstate, switching from one to another working mode, implementing one ormore actions specified in a voice input, and performing other giventasks. For brevity purpose, only one or two tasks are cited whenillustrating voice-related examples below, where other tasks may beapplied without mentioning. Content items presented using or at a devicemay be related to a location, scheduled by a user, arranged by a remotefacility or service center, or specified in a voice input. The contentitems may have video, audio, or another format and may be subscribedwith fees or sponsored by an entity. A device may present content itemsusing a display, a speaker, or other output components. Initially, thedevice may be at a standby, sleeping, power-off, or power-on state. Insome applications, whether or not a user gazes at a device may bedetected. In other applications, whether or not a user gazes at adevice's display, speaker, or another output component may be detected.For brevity reasons, only the former case, i.e., gazing at a device, ismentioned in illustrations below.

When a device is ready, a voice recognition system may be powered on andmonitor a user's voice input via a microphone from the beginning. A gazedetection system may be turned on in response to receiving a user'svoice input. A gaze detection system may also be powered on all thetime.

In both scenarios, a user's verbal instructions are carried out when adevice detects that the user gazes at it. Hence a user's command may notbe carried out, if the user is out of sight, i.e., the user's gazingdirection can't be ascertained. For instance, when a user shouts a fewwords as a command from another room and a device can't find the user insight, the device may not follow the command to do a task even thoughthe device may get the command from a voice recognition system.Similarly, a device may not implement a task if the task is obtainedfrom a voice output generated by another device, such as a television, aspeaker, or a smartphone, since a corresponding gaze doesn't exist andthus can't be detected.

When a name is assigned to a device by default or a user, such as “DJ”,the device may be arranged to perform a task after it receives a voicecommand which contains the name and the task. As used in descriptionsbelow, a name of a device may include a name that is assigned to thedevice and/or a name of a program or app that runs or operates at thedevice. The program or app may be installed at the device optionally.When a name of a device is “DJ”, examples of corresponding voicecommands include “DJ, turn on the lights”. The exemplary commandcomprises the predetermined name and a task and the device may do thetask after receiving the command. Mature voice recognition techniquesmay be used to interpret a voice command. Sometimes, one or twosentences containing a name and a task may come from a television, whenit presents a movie or advertisements. Such a case may be rare, but itdoes have a chance to happen and may become an issue. Thus, there existsa need to avoid taking a voice command from a machine. It may bearranged that a device ascertains whether a voice input comes from auser, after it gets the input which contains a predetermined name and atask. If the device detects that the input is from a user, it performsthe task; otherwise, the device declines to do the task and the inputmay be discarded.

Locating techniques are needed to detect whether a voice input comesfrom a user. For instance, a device may have a locating detector tomeasure the source of a voice and then ascertain whether a target at thesource is a user or a machine. The ascertaining step, or identifyingstep, may be performed using mature identity recognition technologies. Alocating detector may measure and analyze sound waves of a voice inputand then calculate a source position of the voice input via algorithmsusing mature methods. An identity recognition system may use a camera totake pictures of a target at the source position. Whether the target isa user or not may be determined by analyzing the pictures via algorithmsand mature techniques. If the target is a user (or a person), it may beconsidered that the voice input is generated by the user. A device maybe arranged to follow instructions only after it is detected that theinstructions are from a user. When a device receives a voice commandfrom a user who is out of sight, like in another room, the device may beconfigured to ignore or discard the command as it can't determine thecommand is from a user, even though the command contains a name of thedevice and does come from a user.

In some cases, however, we may want a device to control another devicevia a voice command. In some other cases, we may want to tell a deviceto do a task when we are not in sight. For instance, a user may set up awake-up alarm at a smartphone. When the alarm sounds in the morning, italso produces a voice output, like “DJ, turn on the lights”, where DJ isthe name of a device. Then the device may switch on light bulbs in aroom. Sometimes, we may want to shout to issue a command without seeinga device, which means the device can't see us either. In such cases, wewant a device to follow a voice command without ascertaining whether thecommand comes from a user. Thus, there exists a need for a device tofollow a voice command unconditionally, or a need of a specific type ofcommand which a device follows without checking any factors related to auser.

A specific type of command may contain three items: A name, a code, anda task. The name is an assigned name as aforementioned. The codefunctions as a label. When a device receives a voice command containinga predetermined name, it may ascertain whether the voice is from a uservia locating and identification means. When the device detects that thevoice comes from another device, somewhere out of sight, or multiplesources (such as multiple speakers), it may be arranged to decline tofollow the command. In other words, the device may be arranged toimplement the command only when it is detected that the voice comes froma user.

When the device receives a voice command which contains a predeterminedname, a code, and a task, it may follow the command without ascertaininganything related to a user, like whether the command is from a user ornot. A code may be selected and decided by a user. It may be a simpleone which is easy to use and remember. A code may include a numericalnumber, a word, a phrase, a short sentence, or a mixture of numbers andletters. Examples of codes include 123, 225, bingo, listen, “it's me”,and so on. Assume that an assigned name is “DJ” and a code is “it's me”.Examples of voice commands include “DJ, it's me, turn on airconditioning.” When a device receives the command, it gets the name,code and task via a voice recognition system. Since the command has thename and code, there is no need to detect where it comes from or verifyany things. The device may turn on an air conditioning system promptly.

To accommodate various cases and different needs of users, the followingmethod may be arranged. Assume that a device has a voice recognitionsystem for sensing, receiving, and interpreting a voice input. Thesystem is powered on at the beginning. The device also has a gazedetection mechanism or sensor for detecting a user's gaze direction, alocating mechanism or sensor for detecting a source position of a voiceinput, and an identification mechanism or system to detect whether atarget is a user or a machine. The above mechanisms may be inoperational mode from the beginning or triggered individually by asignal after a voice input is received.

Assume a name and a code are assigned to the device. The device receivesa voice input at the beginning. Content of the input may be obtainedthrough the voice recognition system. There are three situations andfive options. In situation 1, it is detected that the voice inputcontains the name, the code, and a task. There is one option, option 1,provided for a user. If option 1 is selected or enabled, the deviceperforms the task right away after receiving the input, since itcontains the name and the code. For instance, a user may say a namefirst, followed by a code, and one or more sentences to describe a taskat last. Aforementioned example “DJ, it's me, turn on the lights” hassuch a sequence along a timeline. Once a device receives the input, thetask is performed without the needs of checking anything else.Alternatively, a user may say the name first, then a task, and finallythe code. For instance, a user may also say “DJ, turn on the lights,it's me”. The code “it's me” is placed behind the task in the sequence.In yet another configuration, a code may come first, like “It's me, DJ,turn on the lights.” When a device receives a voice input, it may searchand recognize three items: a predetermined name, a code, and a task,regardless of a sequence of the items in the input. As long as a devicegets the three items, it is configured to carry out the task when thename and code match a given profile respectively.

In situation 2, the voice input contains the name and a task. There arethree options provided for a user. In option 2.1, the device isconfigured to do the task when the input contains the predetermined nameand the task. In option 2.2, the device is configured to do the taskwhen the input contains the predetermined name and the task and theinput comes from a user. When the device receives the voice input, itmeasures where the voice comes from and then ascertains whether a targetat a source of the voice is a user. If the target is not a user, thetask is not performed. In option 2.3, the device is configured to do thetask when the input contains the predetermined name and the task and itis detected that a user gazes at or looks at the device. When the devicereceives the voice input, it detects the gaze direction of a user. Ifthe user doesn't gaze or look at the device, the task is not carriedout. In addition, the sequence of a name and a task along a timelinedoesn't matter, as long as the name is correct. For instance, a user maysay “DJ, turn off lights” or “Turnoff lights, DJ”. A device may followboth orders and turn off lights.

In situation 3, the voice input contains a task only and doesn't includethe name and the code. There is one option, option 3, arranged for auser. When option 3 is chosen, the device performs the task afterreceiving the input, sensing a user, and determining that the user gazesor looks at the device. The device declines to do the task if it isdetected that the user doesn't gaze or look at the device. In situations2 and 3, when it is detected that a user “gazes or looks at the device”,it means the user gazes or looks at the device when the user issubmitting the voice input or within a given time period after the usersubmits the voice input.

A user may select one, two, or three options each time. For instance, a“Setup” button may be configured on a touch screen of a device. A usermay tap the button to open a setup window, where the user may tap checkboxes to make selections. A user may choose a single one among the fiveoptions to cover one situation only. If a user selects option 1, thedevice performs a task only after it obtains the name, the code, and thetask from an input. If a user selects option 3, the device executes atask only when the user says the task and gazes or looks at the device.

A user may also select two options to cover two situations. Sinceoptions 1 and 2.1, 2.1 and 2.2, 2.1 and 2.3, 2.2 and 2.3, 2.3 and 3overlap each other respectively, there are five possible cases. The fivecases include options 1 and 2.2, 2.3, or 3, options 3 and 2.1 or 2.2. Ifoptions 1 and 3 are selected, for instance, a task is performed when avoice input contains the name, the code, and the task or a voice inputcontains the task and it is detected that a user gazes or looks at thedevice.

In addition, a user may select three options to cover all threesituations. The three selections contain options 1, 2.2, and 3. A taskmay be performed when a voice input contains the name, the code, and thetask, a voice input contains the name and the task and it is detectedthat the voice input comes from a user, or a voice input contains thetask and it is detected that a user gazes or looks at the device.

FIG. 8 is a schematic diagram illustrating embodiments of using multipledevices to implement a command and perform a task according to thepresent invention. Assuming that a user 40 carries a user device 42.User device 42 is associated with user 40 and may include a portabledevice or wearable device such as a smartphone, a smart watch, a smartband, smart glasses, and the like. A control device 44 may implement acommand and perform a task. Control device 44 may also transmitinstructions to another device such as an application device 46.Application device 46 may take instructions from control device 44,implement the instructions, and perform a task indicated in theinstructions. Control device 44 and application device 46 may beconfigured in various settings, such as indoor, outdoor, inside avehicle, etc. User device 42 and control device 44 each may have aspeech recognition mechanism, a microphone, and an identity recognitionmechanism (e.g., a facial or fingerprint recognition system). Userdevice 42 and control device 44 may perform speech and/or identifyrecognition functions using own processors and/or a server at a servicefacility. Assuming that user 40 has logged in user device 42. Controldevice 44 and user device 42 are connected and may communicate with eachother. As such, in some embodiments, an input obtained via user device42 (e.g., an input obtained through a touch sensitive screen of userdevice 42 or a voice input received at user device 42) may be shared byuser device 42 and control device 44, and considered as instructionsprovided for user device 42 and control device 44 by user 40.

User device 42 and control device 44 may be connected and communicatewith each other in various ways. For example, user device 42 and controldevice 44 may be connected by a network (e.g., a Wi-Fi network), aconnection (e.g., a Wi-Fi connection), or a router (e.g., a Wi-Firouter). Once being connected, they may communicate with each other. Inaddition, user device 42 may log in control device 44 directly viaBluetooth or other suitable communication technologies. User device 42may also log in control device 44 indirectly, such as logging in asystem that is connected to control device 44. After user device 42 logsin control device 44 directly or indirectly, the two devices areconnected.

Referring to FIG. 8 . Assuming that application device 46 is atelevision. User 40 may utter a voice input such as “Turn on television”to user device 42. “Turn on television” is a command, and a task aswell. The task may be performed at application device 46. If user device42 is arranged to operate application device 46 directly, user device 42may implement the task. If user device 42 is not arranged to work withapplication device 46 directly, it may send a message to control device44, when control device 44 is configured to do the task. User device 42may extract the task from the voice input using a speech recognitionmechanism. In some embodiments, the message sent from user device 42 tocontrol device 44 may include a text message and a voice message thatcontains data of the voice input (e.g., data of the digital recording ofthe voice input). Alternatively, the message may only contain the textmessage or the voice message.

In some cases, a voice recognition function of user device 42 may beenabled and kept in active state after user device 42 is unlocked byuser 40. Unlocked user device 42 may indicate a system that user 40 haslogged in. As such, any input, e.g., a voice input received at userdevice 42, may be considered as instructions from user 42. Hence, aftercontrol device 44 receives a command to do a task from a text or voicemessage from user device 42, there is no need for control device 44 toauthenticate the command, and thus control device 44 may perform thetask in response to reception of the command. For example, controldevice 44 may not need to verify whether the input is from a person orwhether the input contains a predetermined code before performing thetask.

In some embodiments, the method with reference to FIG. 8 may be combinedwith other methods illustrated above to provide more ways for a user todo a task at a device. As described above, user 40 may utter a commandto control device 44 directly, which may cause control device 44 toperform a task at control device 44 or at application device 46.Alternatively, user 40 may issue a command to control device 44 via userdevice 42 and indirectly order control device 44 to perform a task atcontrol device 44 or at application device 46. For example, options 1,2.2, and 3 as described above may be combined with the method withrespect to FIG. 8 to provide four options for a user. That is, a taskmay be performed by device 44 when device 44 receives or detects a voiceinput that contains a name of device 44, a predetermined code, and thetask, receives or detects a voice input that contains a name of device44 and the task and detects that the input comes from a user, receivesor detects a voice input that contains the task and detects that a usergazes or looks at control device 44, or receives the task from userdevice 42. Optionally, any two or three of the four options may beselected by a user to form alternative combinations as alternativemethods. For example, a task may be performed by device 44 when device44 receives or detects a voice input that contains a name of device 44,a predetermined code, and the task, or receives the task from userdevice 42. For users who want to do a task without using a predeterminedcode and a gaze act, other arrangements may be configured. For example,a task may be performed by device 44 when device 44 receives or detectsa voice input that contains a name of device 44 and the task, orreceives the task from user device 42.

As illustrated above, after user device 42 detects a voice input fromuser 40, user device 42 may send to control device 44 a text message, avoice message, or both the text and voice messages. The text message maycontain a command obtained from the voice input through a speechrecognition method by user device 42. The voice message may contain adigital recording of the voice input. In the first scenario when controldevice 44 receives only the text message, control device 44 mayimplement the command such as performing a task indicated in thecommand. In the second scenario when control device 44 receives only thevoice message, it may obtain or extract a command from the voice messageusing speech recognition and then implement the command. In the thirdscenario when control device 44 receives both the text and voicemessages, there are two options. Control device 44 may implement acommand obtained from the text message. Alternatively, control device 44may get instructions from the voice message via speech recognition,compare the command from the text message with the instructions, andthen implement the command if the command matches the instructions, orimplement the instructions when the command and the instructions do notmatch.

In some embodiments, during or after implementing a command orperforming a task, control device 44 may transmit a reply message touser device 42. The reply message may work as a summary or report thatis a response to reception of the text and/or voice message from userdevice 42. The reply message may include the command and/or task (e.g.,a name of the task or descriptions of the task) performed via device 44.The reply message may also indicate device 44 performed the task oranother device (e.g., application device 46) performed the task, andinclude time information about the command and task, and locationinformation when location data is available. The time information maycontain a time around which the command and/or task is performed. Thelocation information may contain location data of control device 44and/or application device 46. User device 42 may store user data such ascommands and tasks performed via control device 44, the timeinformation, and the location information. The user data may be storedat user device 42 and/or a service facility. User device 42 may use theuser data to facilitate interpretation of a voice input when the voiceinput contains an incomplete command. For example, user 40 may utter“Television” as a verbal command, or in another case, only one word“television” may be recognized from a voice input. As “television” onlyrepresents a keyword of a command or task, the verbal command indicatesan incomplete command, and cannot be treated as an executable command byuser device 42 and control device 44 using conventional recognitionmethods. However, if user device 42 has user data including past voiceinput received from and past commands performed for user 40, user device42 may find that user 40 submitted tasks such as “Turn on television”and “Turn off television” respectively at least a number of times withina period of time (such as one to six months). Hence, user device 42 maysend a message to control device 44 and request control device 44 toturn on the television if it is off or turn off the television if it ison. Hence, records of past commands and activities may be utilized byuser device 42 to convert an incomplete command into a suitable commandwhen the incomplete command only contains one or a few keywords. In somecases, control device 44 may also store data of user activities and usethe data in similar ways to process incomplete commands when permittedby user 40.

In some cases, user device 42 may receive or detect a voice input fromuser 40 and control device 44 may receive or detect a voice signal inthe same time period. Thereafter, user device 42 may transmit a textand/or voice message containing a first command to control device 44.The first command is from the voice input and speech recognition may beused in interpretation by user device 42. In the meantime, controldevice 44 may obtain a second command from the voice signal throughspeech recognition. After receiving the message from user device 42,control device 44 may get the first command from the message and comparethe first command with the second command to obtain a comparison result.If it is determined that the first and second commands are the same orsimilar, control device 44 may implement the first or second command. Ifthe first command is incomplete and the second command is a completecommand, control device 44 may implement the second command. If thesecond command is incomplete and the first command is a completecommand, control device 44 may implement the first command. If the firstcommand and the second commands are two complete but different commands,there are two options from which a user may select. In the first option,control device 44 may implement the first command. A message may bedisplayed on a screen of user device 42 to show the two commands andinform user 40 that the first command is performed, while the secondcommand is not implemented. In the second option, control device 44 mayimplement the second command. A message may be displayed on the screenof user device 42 to show the two commands and inform user 40 that thesecond command is performed, while the first command is not implemented.

Optionally, after receiving a voice input and before transmitting a textand/or voice message containing a command to control device 44, userdevice 42 may authenticate user 40. In some embodiments, a text and/orvoice message containing a command may be transmitted from user device42 to control device 44 only after user 40 is identified or recognized.User device 42 may use a facial recognition mechanism to recognize user40. Alternatively, user device 42 may have a voice recognition systemthat may be arranged to verify that the voice input matches specificfeatures (e.g., voice print information) of user 40's voice and theverification result may be used to identify user 40. In addition, userdevice 42 may identify user 40 by a fingerprint verification method.Optionally, user 40 may enter a password or passcode at device 42 to getrecognized. By identifying user 40 before transmitting a command tocontrol device 44, it may protect the privacy of user 40 and prevent anunauthorized user from accessing control device 44 and performingcertain tasks.

FIG. 9 is a schematic diagram illustrating embodiments of using a userdevice for receiving and transmitting a command according to the presentinvention. Assuming that user 40 carries user device 42. Device 42, asillustrated above, is associated with user 40 and may include a portabledevice or wearable device such as a smartphone, a smart watch, a smartband, smart glasses, and the like. A device 48 may include an electronicdevice, a machine, or a vehicle. The machine may include an electronicdevice that is used to serve the public and fixed at a location, such asa vending machine installed in a shop or outside a shop. The vehicle maybe a driver-operated vehicle or an autonomous vehicle (also known as adriverless or self-driving vehicle). The vehicle may include anautomobile, a drone (or unmanned aerial vehicle (UAV)), an aircraft, aflying car, a ship, or a motorcycle. In descriptions below, anautonomous automobile is used as an example for device 48, and device 48is referred to as vehicle 48.

Vehicle 48, as an autonomous automobile, may include a vehicle controlsystem and a driving system responsible for vehicle navigation anddriving, respectively. The vehicle control system may include aprocessor and a computer readable medium. The processor may run programsor sets of executable instructions stored in the computer readablemedium for performing various functions and tasks, e.g., receiving andprocessing data collected from sensors, communicating with a servicecenter 83, retrieving map data from the medium or service center,sending driving signals to the driving system, monitoring,communicating, and interacting with a user, executing otherapplications, etc. The vehicle control system may also include input,output, and communication components.

Vehicle 48 may also include a speech recognition mechanism, amicrophone, and an identity recognition mechanism (e.g., a facial orfingerprint recognition mechanism) for performing speech recognition andidentify recognition, respectively. User device 42 and vehicle 48 may beconnected and communicate with each other in various ways.

In some embodiments, user device 42 and vehicle 48 may be connected toservice center 83, respectively, via communications networks. Forexample, an app, which may be referred to as Car App, may be installedat user device 42. Car App may provide functions for a user to hail avehicle, place a purchase order, receive a package from a deliveryvehicle, etc. After user 40 opens Car App, the app may communicate andkeep connected with service center 83. As vehicle 48 is connected withservice center 83 continuously, vehicle 48 may communicate with userdevice 42 (i.e., Car App) via the service center. Optionally, when userdevice 42 and vehicle 48 are within a certain distance, they may beconnected directly by, for example, a network (e.g., a Wi-Fi network), aconnection (e.g., a Wi-Fi connection), or a router (e.g., a Wi-Firouter). Once being connected, user device 42 and vehicle 48 maycommunicate with each other.

After Car App is launched, the interface of Car App appears on a touchscreen of user device 42. User 40 may check the status of a vehiclehailed, an order placed, or an incoming package via the interface of CarApp. When user device 42 detects inactivity for a certain time period,the screen of user device 42 may turn dark, and the device enters alocked state and standby state. The standby state may end when user 40unlocks the device by, e.g., a fingerprint method, facial recognitionmethod, or entering a password. However, after a user hails a vehicle orplace an order (e.g., a takeout order from a restaurant) and a selectedvehicle is on its way to the user, the user may want to check the statusof the selected vehicle frequently. As unlocking a device takes certaineffort, a shaking act may be used to resume Car App and display theupdate of the selected vehicle in a simpler way.

For example, Car App may have a shaking mode. When the shaking mode isenabled, a detector (e.g., a detector similar to detector 24 of FIG. 1 )may be arranged in an active state and keep monitoring and sensingshaking acts when user device 42 is locked and on standby. Optionally,other acts such as tapping the screen of user device 42 may also beincluded and have the same effect as the shaking act. Provided Car Appis not closed when the user device enters the standby state. In such acase, the state of Car App may also be considered as a standby statewith reduced functions and lower power consumption. In response todetection of a shaking act or shaking of user device 42 during thestandby state, user device 42 may resume or reactivate Car App andpresent the interface of Car App on the screen. For example, user 40 mayshake user device 42 lightly for a couple of times, such as 2 to 5 timesalong any direction. After user device 42 senses the shaking act, userdevice 42 presents the content of Car App on the screen. That is, CarApp resumes activities with the interface showing, for example, thestatus and a current location of the selected vehicle, which may benavigating toward the user. Car App also displays updated information inthe interface after receiving updates from the selected vehicle and/orservice center 83. A moving item representing the approaching selectedvehicle may also be shown on a map covering nearby areas in theinterface.

When user device 42 detects that it is shaken more times than a givennumber, e.g., more than 5 times, the shaking act may be ignored, as itmay mean something else happened. As the identification step may beskipped for the convenience to view the updates, user device 42 may bemaintained in the locked state to protect personal information, eventhough the interface of Car App returns and Car App is active inoperation. Optionally, only when a selected vehicle (e.g., vehicle 48)is within a certain distance (e.g., 1-2 miles) from user 40 or a drivingtime of a selected vehicle is less than a certain value (e.g. 10-15minutes), the shaking act may reactivate Car App and cause its interfaceto reappear on a dark standby screen of user device 42. Thus, when aselected vehicle is not available or farther than a certain distance, ashaking act does not reactivate Car App, which may prevent unnecessarypresentations and information leaks. As such, user 40 may view Car Appwithout unlocking user device 42 when a selected vehicle is coming. Themethod may provide certain convenience for checking the status of aselected vehicle. User device 42 may receive information (e.g., locationinfo) of a selected vehicle from service center 83 or directly from thevehicle.

As Car App contains personal data, the shaking mode may cause privacyconcerns. To reduce risks, a button, such as a shaking mode button, maybe configured in the interface of Car App. User 40 may tap the button toenable the shaking mode. Then, when user device 42 and Car App are onstandby and user 40 shakes the user device a bit, the user device maydetect the shaking act and reactivate and present Car App in response.If Car App is closed or is not open before the standby mode, user device42 may ignore the shaking act and not reactivate Car App. Optionally,once Car App is closed, the user device may terminate the shaking modeto avoid accidental exposure of the program. Optionally, after a certaintime ((e.g., 15-30 minutes) of inactivity with regard to Car App, CarApp may request user device 42 to terminate the shaking mode to protectthe user privacy.

In some cases, a mike button representing a voice mode may be configuredin the interface of Car App. After user 40 taps the mike button, thevoice mode is enabled and the user may speak to communicate with Car Appor the selected vehicle via user device 42 verbally. Provided Car Apphas a name such as “App”, the selected vehicle has a name such as“Vehicle”, and the voice mode is enabled. When user device 42 is in anunlocked state, and Car App is active with its interface shown on thescreen, user 40 may utter to user device 42 a command or a questiondirectly without saying a name (e.g., “App” or “Vehicle”). The userdevice detects the voice input, converts it into a text message using avoice recognition technique, and then transmits the text message to CarApp. In response, Car App may implement the command or answer thequestion by showing a text message or generating an audible output. Ifthe question is for the selected vehicle, Car App may forward it to thevehicle and present an answer to user 40 after obtaining it from thevehicle. For example, after user 40 utters “What is waiting time” or“Tell me name of the sender”, Car App may obtain an answer based oninformation collected, or get an answer from vehicle 48. Thereafter, CarApp may display the answer in the interface.

Optionally, when the voice mode is enabled, user device 42 is in alocked mode, and both the device and Car App are on standby, user 40 mayutter a command or question to user device 42 that may pass the info toCar App. In such cases, the user may include a select name (e.g., “App”or “Vehicle”) in the voice input along with a command or question. Theselect name may be determined by service center 83 and presented to theuser. As such, the microphone of user device 42 may be set in an activestate and monitor any voice input continuously when user device 42 is onstandby. After the microphone detects a voice input, user device 42 mayconvert the input into a text message and determine whether the inputcontains the select name. If the select name is included in the message,the user device may send the text message to Car App. Next, Car App mayreact to the voice input, implementing a command, presenting a textmessage on the screen, or answering a question audibly. Alternatively,the screen of user device 42 may remain dark when Car App responds tothe request of the user. If user device 42 does not detect any selectname in a voice input when the user device is locked, the input isignored or discarded. Hence, two options, corresponding to an unlockedmode and locked mode of the user device, are provided for a user tosubmit verbal instructions or questions. It may facilitate convenientaccess of Car App, while preventing miscommunication and unintendedcommands. To protect the privacy of users, the voice mode may beterminated after Cap App is closed or terminated in some cases.

After user 40 hails a car or place an order such as a takeout orderusing Car App, a vehicle (e.g., vehicle 48) may be selected by servicecenter 83. The selected vehicle is dispatched to pick up the user or thetakeout. In the latter case, the selected vehicle will make a deliveryto the user. Assuming that the user chooses or accepts roadsidedelivery. The term “roadside delivery” as used herein indicates that apackage is delivered to a user at a location by the roadside (or atcurbside), at a parking lot, or in a driveway. In such a case, a sectionof the roadside, the parking lot, or the driveway is proximate or closeto a place of a delivery address. The user may go outside to meet withthe selected vehicle and get a package. Compared to conventional methodsthat deliver a package to the doorstep, roadside delivery is moresuitable for autonomous vehicles and may have a lower shipping fee.

Vehicle 48 and Car App may keep communicating and exchange locationdata, especially when vehicle 48 approaches user 40 or user 40approaches vehicle 48. As such, user 40 may communicate with vehicle 48via Car App before seeing it. For example, user 40 may utter a commandto Car App through user device 42 and let Car App pass the command tovehicle 48, instead of getting very close to the vehicle and thenfinding an interface device (e.g., a microphone or touch screen) forcommunication. Both Car App and vehicle 48 may use the location data tocalculate the distance between vehicle 48 and user 40. Further, thevehicle may use sensors (e.g., a camera) and the location data to finduser 40. After finding user 40, vehicle 48 may keep monitoring the userusing cameras and microphones. The vehicle may measure the distancebetween vehicle 48 and user 40 using an optical method (e.g., atime-of-flight method) and send the distance data to Car App. Themeasured distance data may overwrite the data obtained by calculation.If user 40 wants to go to a place, the user gets in the vehicle and thenproceeds with check-in procedures before a trip gets started.

If vehicle 48 is a delivery vehicle, and carries a parcel or package tobe delivered to user 40, Car App may present a question to getpermission for releasing the parcel. The question may be displayed inthe interface of user device 42. Car App may be active with itsinterface shown on the screen of the user device. Car App may also be onstandby along with locked user device 42. In either of above scenarios,when the distance between vehicle 48 and user 40 is smaller than acertain value (e.g., 2-5 meters), Car App and/or vehicle 48 may ask theuser for consent to present and release the package to the user. Forexample, vehicle 48 may transmit a message to Car App and prompt Car Appto display a question in the interface of Car App, such as “Receivepackage now?” or “Get package now?” or “Release package now?” Indescriptions below, the verb “receive” is used as an example.Optionally, vehicle 48 may also present the question on a display (notshown) of the vehicle. The display may be mounted on the exterior of thevehicle. Provided user device 42 monitors the voice input of user 40 viaa microphone continuously. User 40 may tap a yes button in the interfaceof the user device, utter “Yes” to the user device 42, or utter “Yes” tothe display of the vehicle. User device 42 may detect the tapping act orthe verbal answer, and then transmit the reply to vehicle 48. Thevehicle may also detect the verbal answer when the user is close enough.When the answer is verbal, user device 42 and/or vehicle 48 may convertit into a text message using a speech recognition technique. Optionally,user device 42 may make a digital recording of the verbal input and sendthe recording to vehicle 48. The vehicle then translates the recordinginto a text message.

In response to a positive reply (e.g., “Yes”) of user 40, the controlsystem of vehicle 48 opens a compartment such as a compartment 50 at thevehicle. User 40 then may take a package from the compartment.Optionally, vehicle 48 may detect the user and confirm that the user isin front of the vehicle before releasing the package. If the packagecontains a takeout sent to user 40 from a restaurant and vehicle 48 hascertain information of the takeout from the restaurant or service center83, the question displayed may include one or more words that indicate ameal or foods prepared at the restaurant. Mentioning a meal from arestaurant may motivate a user to receive it immediately when it isstill hot, which may reduce the delivery time for vehicle 48. Forexample, the question may be “Receive pancakes now?” or “Receive takeoutnow?” Optionally, the question may also contain one or more words thatcorrespond to a name of a restaurant or indicate a restaurant, such as“Receive meatball from A's Kitchen?” or “Receive order from A'sKitchen?” which also motivates a user to get the package quickly. Whenthe question contains one or more words indicating a restaurant, thequestion may not need to have words that reflect a meal or foods fromthe restaurant.

If user 40 is busy at the moment, the user may tap a “Later” button inthe interface of Car App, utter “wait” to user device 42 or the vehicle,or does not reply. No reply may mean not ready to accept the package.The vehicle may wait for user 40 for a given time period. When user 40wants to have the package after a while, the user may come back and keyin a passcode at vehicle 48 to be identified. The user may also tap a“Receive Parcel” button in the interface of Car App, or utter a key wordsuch as “parcel” as a command to user device 42 when the interface ofCar App is displayed and Car App is active. If user device 42 and CarApp are both on standby, user 40 may utter to user device 42 Car App'sname and a command, such as “App, parcel please”. User device 42 maydetect the name and command and transmit the command to Car App, whichthen passes the command to vehicle 48. Next, vehicle 48 may detect thelocation of user 40 and open a door of a compartment to release apackage when the user is within a certain distance from the vehicle.

CONCLUSION, RAMIFICATIONS, AND SCOPE

Thus it can be seen that systems and methods are introduced forpresenting information and performing a task at an electronic device, amachine, or a vehicle.

The improved methods and systems have the following features andadvantages:

-   -   (1). An idle or standby device is used to present content items        to a user;    -   (2). Gazing direction is used to determine when to present        content items and when to stop it;    -   (3). User input such as shaking, tapping, or speaking to a        device is combined with a gaze detection to determine when to        present content items;    -   (4). Detection of a name, a code, and/or a gaze is used to        determine when to perform a task;    -   (5). A user input at a device is transmitted to another device        for the other device to perform a task;    -   (6). A user device is utilized for communication between a user        and another device, a machine, or a vehicle; and    -   (7). A locked and standby user device is utilized for        communication between a user and another device, a machine, or a        vehicle.

Although the description above contains many specificities, these shouldnot be construed as limiting the scope of the invention but as merelyproviding illustrations of some of the presently preferred embodiments.Numerous modifications will be obvious to those skilled in the art.

RAMIFICATIONS

A presentation method based on eye-tracking or gaze-sensing technologiesmay be applied to cell phone, smart phone, smart watch, tablet computer,laptop computer, desktop computer, television, game player, digitalbillboard, or any other electronic devices or systems having a displayand certain computing power.

Ambient light sensor may be added to a device to sense ambient lightintensity, which may be used to determine whether the device is in apocket or bag. If a device is not pulled out, measurement results of amotion sensor may be ignored in applications discussed.

A content window may be configured to close by itself when certainmotion is detected by accelerometer or gyroscope sensors, even though auser is still watching the screen, as it is uncomfortable to view anycontent, or inappropriate to show any content in such conditions.

Moreover, a device may be equipped with a facial recognition system tocreate an extra layer of protection. The system may at least recognize adevice owner, which may protect a user's privacy by not following otherpeople's instructions, or may be used to present different informationto different users according to prescheduled plans. For instance, thesystem may be used to identify a user against given facial criteria. Ifan identification process fails to provide a positive result, any inputreceived from the user may be discarded. No matter what the user does,an operational state or inactive state of a device is not affected bythe user's action. It also means that a user has to be in sight so thata device may ascertain the user and perform an identity verificationprocess. The system may make use of a camera which is employed by gazedetection to get dada and employ facial recognition algorithms toidentify a user.

To trigger a content window by a gazing act, a user may also look atthings located outside a display but close to its edge, instead oflooking at the display directly. The reason is that, when a user looksat objects close to a display, content shown on the display may alsoreach the eye, thus providing a viewing opportunity anyway. Andhopefully, the user may turn his or her sight a bit to get a betterreception of the content. Moreover in many cases, instead of display, itmay be enough to trigger a content show if a user just looks at anidling device for a given period of time, because it may mean bothparties are available and the user may have a good chance to noticecontent items displayed on the device. In cases of smartphone and tabletcomputer, gazing at a device is almost equivalent to gazing at adisplay, because for these devices, a display may covers the whole areaof one side.

Lastly, a method may be configured which ascertains whether a user facesa device, instead of gazing at a device. In some applications, it may bedifficult to sense a user's eye movement, due to technical issues orambient lighting conditions. Thus it may be arranged to detect whether auser faces a device. For instance, a device may use an imaging sensorlike camera to take pictures or videos of a user. Certain algorithms maybe used to identify facial features of the user, determine positions ofthe user's eyes, and then calculate a distance between a spot of thedevice and one eye and another distance between the spot and the othereye. The spot may be a point at the center of the device or the centerof an output component. If difference of the two distances is smallerthan a given value, it may be considered that the device is right infront of the user or the user faces the device. Consequently, it may beconfigured that in all of above discussions, a gazing requirement may bereplaced by a facing requirement when a user or entity decides to do so.For instance, a requirement of gazing at a device may become arequirement of facing a device.

Therefore the scope of the invention should be determined by theappended claims and their legal equivalents, rather than by the examplesgiven.

The invention claimed is:
 1. A method for an electronic device,comprising: 1) detecting an act made by a user involving physicalcontact with the electronic device or physical movement of theelectronic device when a display of the electronic device has an idlescreen or a screen in standby mode, inactive mode, or screen-saver mode;2) performing gaze detection only after detecting the act; and 3)ascertaining whether the user looks at a direction toward the electronicdevice using a result of the gaze detection, wherein a result ofascertaining whether the user looks at the direction toward theelectronic device is used to determine whether to perform a task at theelectronic device.
 2. The method according to claim 1 wherein the taskincludes presenting a plurality of content items on the display.
 3. Themethod according to claim 2 wherein the plurality of content items isunrelated to content presented before a state of the idle-screen, thestandby mode, inactive mode, or screen-saver mode.
 4. The methodaccording to claim 1, further including determining whether the user isrecognized via a recognition mechanism and performing the task after itis ascertained that the user looks at the direction toward theelectronic device and the user is recognized via the recognitionmechanism.
 5. The method according to claim 4 wherein the recognitionmechanism includes a facial recognition method.
 6. The method accordingto claim 2 wherein the plurality of content items includes at least oneof the following items: news, alert, update, email, text message,sponsored content, and other information.
 7. The method according toclaim 4, further including performing a different task when a differentuser is recognized.
 8. A method for an electronic device, comprising: 1)monitoring the electronic device or physical movement of the electronicdevice to sense an act made by a user when a display of the electronicdevice has an idle screen or a screen in standby mode, inactive mode, orscreen-saver mode; 2) performing gaze detection only after sensing theact; and 3) detecting whether the user looks at a direction toward theelectronic device, wherein a result of detecting whether the user looksat the direction toward the electronic device is used to determinewhether to perform a task at the electronic device.
 9. The methodaccording to claim 8 wherein the task includes presenting a plurality ofcontent items on the display.
 10. The method according to claim 9wherein the plurality of content items is unrelated to content presentedbefore a state of the idle-screen, the standby mode, inactive mode, orscreen-saver mode.
 11. The method according to claim 8, furtherincluding determining whether the user is recognized via a recognitionmechanism and performing the task after it is detected that the userlooks at the direction toward the electronic device and the user isrecognized via the recognition mechanism.
 12. The method according toclaim 11 wherein the recognition mechanism includes a facial recognitionmethod.
 13. The method according to claim 9 wherein the plurality ofcontent items includes at least one of the following items: news, alert,update, email, text message, sponsored content, and other information.14. The method according to claim 11, further including performing adifferent task when a different user is recognized.
 15. An electronicdevice, comprising: one or more processors; a memory; and one or moreprograms, wherein the one or more programs are stored in the memory andconfigured to be executed by the one or more processors, and when theone or more programs are executed by the one or more processors, the oneor more processors are caused to perform: 1) detecting an act made by auser involving physical contact with the electronic device or physicalmovement of the electronic device when a display of the electronicdevice has an idle screen or a screen in standby mode, inactive mode, orscreen-saver mode; 2) performing gaze detection only after detecting theact; and 3) ascertaining whether the user looks at a direction towardthe electronic device using a result of the gaze detection, wherein aresult of ascertaining whether the user looks at the direction towardthe electronic device is used to determine whether to perform a task atthe electronic device.
 16. The electronic device according to claim 15wherein the task includes presenting a plurality of content items on thedisplay.
 17. The electronic device according to claim 16 wherein theplurality of content items is unrelated to content presented before astate of the idle-screen, the standby mode, inactive mode, orscreen-saver mode.
 18. The electronic device according to claim 15wherein the one or more processors are further caused to determinewhether the user is recognized via a recognition mechanism and performthe task after it is ascertained that the user looks at the directiontoward the electronic device and the user is recognized via therecognition mechanism.
 19. The electronic device according to claim 18wherein the recognition mechanism includes a facial recognition method.20. The electronic device according to claim 16 wherein the plurality ofcontent items includes at least one of the following items: news, alert,update, email, text message, sponsored content, and other information.